Various techniques for recognizing a state of a crowd (which will be denoted as crowd state below) in an image are proposed (see PTLs 1 to 3).
A person behavior determination device described in PTL 1 extracts a changed region in which a difference is caused by background differencing or the like from a video, and calculates the characteristic amount from the changed region. The person behavior determination device then determines whether the changed region is a person region by use of a person discriminator machine-learning the characteristic amount, thereby detecting the person region. Further, the person behavior determination device associates the detected person region between frames in consideration of distance or color histogram, and tracks the person region over a predetermined number of frames. The person behavior determination device then calculates the characteristic amount of a person trajectory such as average speed, tracking time and motion direction from the person trajectory acquired by the tracking, and determines a person behavior based on the characteristic amount of the person trajectory.
A headcount counting device described in PTL 2 counts the number of persons from a video shooting a crowd therein. The headcount counting device extracts the heads of persons included in the image based on head models. The headcount counting device then links the head positions determined as the same person between frames by use of the characteristic amount such as position information or color distribution, and counts the number of persons from the linking result (person tracking result).
A system described in PTL 3 detects a state such as steady (main stream of persons, for example)/non-steady (against main stream, for example). The system aggregates optical flow attributes for a determination block as a determination unit, and calculates an evaluation value for evaluating a degree of steadiness of optical flow. The system then determines a state of the determination block from the evaluation value.